Add Android sample (JNI binding + Java wrapper class) #101
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds
examples/clip.android
, an Android sample that interfaces with the shared libraries ofclip.cpp
through JNI. The structure of the project is highly inspired fromllama.cpp/examples/llama.android
.Summary of changes
examples/clip.android
is an Android project with two modulesapp
andclip
.app
module contains the sources for a demo application which allows the user to select an image and enter a text description to compare them (uses Kotlin/Compose)clip
module containsclip/src/main/cpp/clip-android
that contains the JNI interfaces which allow communication with the underlying functions fromclip.h
.clip
module containsclip/src/main/java/android/clip/cpp/CLIPAndroid.java
which is wrapper class containingnative
methods.clip
module also houses an instrumented testclip/src/androidTest/java/android/example/clip/CLIPInstrumentedTest.kt
written in KotlinREADME.md
is included in theexamples/clip.android
directory which describes how to run the sample app and the instrumented testclip.cpp
is modified to address issue (Possible) Incorrect assignment of image mean/std values inclip_model_load()
#99CMakeLists.txt
is modified to add-mcpu=native
when${CMAKE_SYSTEM_PROCESSOR} MATCHES "aarch64"
only if-DCLIP_NATIVE
is enabledclip_image_preprocess_no_resize
inclip.h
andclip.cpp
which does works exactly likeclip_image_preprocess
but does not perform linear interpolation (no image resizing). Android'sBitmap
class and Python's PIL can handle bilinear interpolation easily, so it would be great to have a method that skips resizing.Note
Though the
clip
module here in built as an Android library module, it can be built into a 'Java library' module (packaged as a JAR) and used in other Java-only projects as a port toclip.cpp
It was fun working on this sample, and I would like @monatis to review it and share feedback.